SbCOHENT RESUHE 



ED 111 370 



IB 002 424 



AUTHOR 
TITLE 

PDB DATE 
NOTE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Boles^ Suzanne 

A Conceptual Framework for Program Planning and 
Evaluation Within Public Libraries. 
Jul 75 

lOp.; Paper presented at the American Library 
Associa:ion Annual Co iference (94th^ San Francisco^ 
California^ June 29 through July 5, 1975) 

MF-$0.76 HC-$1.58 Plus Postage 

Conference Reports; ♦Evaluation Criteria; *Library 
Programs; Library Services; Objectives; *Performance 
Specifications; Program Planning; ♦Public 
Libraries 
ALA 75 



ABSTRACT 

In public library program plann 
be placed on the linkage of measurement to eval 
assumptions underlie program planning and evalu 
committment must be made to develop goals and o 
measurement must be taken in regard to specific 
evaluation must be implemented. Performance or 
provide a comparison between what is intended a 
and the data obtained must be used to alert man 
in staff philosophies and to determine if the s 
(Author/DS) 



ing^ emphasis should 
uation. Three basic 
ation: (1) a basic 
bjectives^ (2) 

criteria, (3) a final 
program targets 
nd what is achieved 
agement to differences 
ervice is effective. 



* ♦♦♦♦♦♦♦♦♦♦♦♦ ♦♦♦♦:^c*********** ******* :|t 

* Documents acquired by ERIC include many informal unpublished * 

* materials not available from other sources. ERIC makes every effort * 

* to obtain the best copy available, nevertheless, items of marginal * 

* reproducibility are often encounte.^id and this affects the quality * 

* of the microfiche and hardcopy reproductions ERIC makes available * 

* via the ERIC Document Reproduction Service (EDRS) . EDRS is not * 

* responsible for the quality of the original document. Reproductions * 

* supplied by EDRS are the best that can be made from the original. * 
************************************* ***^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 



o 

t— f 



ci5 

0 

0 



A OONCEPTUftL FRMESflORK FOR 
PROGRAM PLWSNING MD EVALUATION 
WITHIN PUBLIC LIBRARIES 
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Luf This paper examines the p2X)cess of program planning and evaluation 

and its inter-relationship to performance measures, with enph^-^is on the 
linkage of measurement to evaluation. It is a conceptual framework for 
program planning and evaluation within the real world of public libraries. 
Program planning and evaluation is a process which requires policy 
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makers "to jointly identi^. . .catron goals, define eech individual's major 

area of responsibility in terms of results expected ^ and use these 

•measures' for operating the library and assessing the contributions of 

eadi of its members. It "serves as a systematic process for determining 

organizational direction and evaluating results through the identification, 

clarification, and asirnrunication of mutually accepted and carefully defined 
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goals and objectives througbDUt the library." 

It is inportant to begin by describing the process and defining the 
terms used to denote the various phases of the process. The basis of 
planning is an understanding of the grand design — a statement of the 
values held by the institution. The first element in plannii>g is identi- 
•fication of a need. VJhat situation requires attention? Is there a 
problem or issue that requires a new approach? What evidence exists 
that the situation or problem is actual and significant? What one 
C]J intends to acconplish is tne goal; to what end will action be directed? 

Ihe d^jective, vMch is the guide for action, describes in specific terms 
what needs to happen to fulfill the intentions. Itow the objective is reached 
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should be determined by identifyiiig possible ways of meeting the need, 
then arriving at a decision of the most feasible alternative. An action 
plan beoxnes the next element; who will do VN*at, vAien, and with vAiat 
resources or help? The last element of planning begins the transition to 
evaluation. At this point consideration must be given to hGW one is to 
knew if the various actions were taken and if they were effective. What 
results are important? IkM will these results be determined? The speci- 
fications of these outcomes are, performance or program targets. They 
provide a focus for irteasurement of expectations against actual results — 
a focus vMch highlights the most significant areas of the phenomenon under 
consideration. Performance or program targets can be both qualitative and 
quantitative, dependant upon the aspect of the program beii^ measured. 
Criteria for evaluation, methods and procedures for data collection and the 
questions asked ccxT?)rise the diinensions of nmsures used to find whether the 
targets are being reached. It is the linking of "performance measures" 
and their analysis to the performance targets that is iirperative if evalua- 
tion is to take place. Ihis is the crucial task. 
The Program Planning and Evaluation Process 

There are three basic assunptions vMch underlie program planning 
and evaluation. First, participants in the process must "have a basic 

cximiitment" to develop "service goals and objectives which will meet 
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sore of the unfulfilled needs of the people." Otherwise, planning is 
no more than a symbolic gesture; the planning document, rot the achieve- 
ment of goals, becones the "final arbiter."^ 

Second, any "assessm2nt of performance is linked with declared 
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objectives." Ihe consequences of decisions made in the PP&E process 
must be measured agaixisc specific criteria. Hcjw is it known if an 
appropriate level of performance has been reached? Hew is it known if 
success, even progress toward the objectives, has been achieved? 

Ihe third assurtption is that the purpose of evaluation is to provide 
facts to assist decision makers determine if goals and objectives are being 
met. Can they insure the value of the service to the public or is there 
a discrepancy between intentions and outcomes of the service? It is essen- 
tial for facts to be torthccming throughout the process in order that the 
service can be iiiproved or modified, if necessary, on a continuous basis. 
A final evaluation document is important, but not so irtvortant as the 
pec^le v*io may have been poorly served during its preparation. 

In setting goals and objectives and steps necessary to iitplenent 
thOT, planners mast be ccnpetent and willing to determine the variety of 
goals vMch will influence the process. Openness about all the goals is 
demanded if later confusion or problans are to be lessened. Mninistrators 
may have a set of goals vdiich may or may not be identical with those of the 
institution. Planners may want to further the possibility of job advance- 
ment or security by participatir^. Recipients of the service may have 
- needs inconpatible with v*iat the institution thinks it should meet. To 
assume that the goals detemined by the planners, vAx> are already convinced 
of the need for the service, are the only ones in existence or will be 
readily accepted by all, can only lead to confusion and resistance vAien 
changes or modifications are required in the service. 

Just as goals must be openly understood and attainable, objectives 
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should be specific statennents derived from the goal, measureable by 
seme criteria. Unnfeasured conclusions and unsupported recoimendations will 
not aid decision-making. It is specification of these objectives which 
beocmes the means to evaluation. The "details of the service and the 
object of the service, the user population," must be known before appro- 
priate criteria can te determined.^ Key questions can be drawn fron the 
hoped-for results. These questions form the basis for conparing intention 
against performance. 

The Relationship of Performance Measures to PP&E 

Performance measures, or measured outputs hiave the significant role 
of substantiating intuitive feelings about a service or program of an 
ipstitution. They can offer concrete data to articulate perceptions about 
results of the service. Measurement is a primary tool for ordering a process 
by v.'hi.ch v.-e gain understanding of the ccnrranality of various eleirents with 
greater precision and discrimination. Just because the maasures may be 
quantifiable does not eliminate interpretation of the value of the thing 
being measured. In fact, vdien measuring the worth of a service to a user, 
numbers must act as means to an end. If questions to users are in a fashion 
requiring them to state an opinion or feeling about a service, there still 
must be enough of these opinions—numbers again—to give an adequate evaluation 
of the service. How one determines whether an aspect of public library 
service is measureable in the first place depends, of course, on how that 
seirvice is viewed by those involved. 

Foundations for arriving at performance measures are found in the 
evaluative criteria dealt with throughout the PP&E process—examination of 



program alternatives; selection of the most feasible course, determination 

of action steps and" setting performance and program targets. Let's 

explore these evaluative criteria. 

Probing of alternatives for accoiplishment of the objectives oi the 

service is a difficult and often frustratix>g job. One revolves between 

non-reality and real-world constraints, between dream-wrld courses of 

action and limited resources. In a group of people, a variety of ideas 

and differences among v*iat each views as possible can emerge. But in 

setting forth specific ways to proceed, one is examining the criteria of 
« 

the potential service and judging the value of each alternative. Once 
non-reality and reality merge and the most feasible way to proceed is 
realized, the series of steps vMch focus on does what, when, needs 
to be formulated. Each step is a basis for evaluation— to determine what 
is being dons, and if it is not, vAiy not. 

A logical extension of these criteria is the setting of performance 
or program targets, to have sonnething discrete by vAiich to measure vdiat 
you hope to achieve. As stated earlier, they provide a focus for corparison 
between vAiat is intended and vAiat is achieved. They lead to the estimation 
of key result areas or outcones of the service. 
Measurement to Evaluation 

One now arrives at determination of the measurement criteria or 
performance data. What do you need to know fran whan, by when, by what 
kinds of forms or interviews or other research methods. Performance measure- 
ment is the process of getting vAiat you need to know. If the quality of a 
service is to be adequately and honestly judged, the measurement process 



must occur. This is vdiere one ccmes full force against the "real wrld." 
Cries of "Where's the service?" come forth fran staff when orphasis is 
necessarily placed on trainiiig non-researchers, the librarians vdx> work 
directly vdth the patrons, to create the forms and gather the information. 
But if a library is ccmtnitted to the concept that research fron the "inside", 
e.g., by staff rather than researchers unaffiliated with the institution, 
can be viewed objectively and can be more inf luencial with ttose who have 
the responsibility for making decisions, the tims required and the concen- 
tration on helping th*e practicing librarian understand the reasons for and 

« 

techniques of data collection must be accepted. It must be remembered arxi 
stated, over and over, that the point of neasxaranent is to determine the 
quality of the perfomonce of a service. 

t 

To discover to v^iat degree of effectiveness a service is being per- 
formed, e.g., to what degree it is meeting the needs of people, those 
providing the service and those receiving it must be questioned. If this 
information is collec::ed at only a few points, by a few cortnitted individuals, 
the task is easier than v^en a large number of people are involved; easier, 
but not as valuable to the institution in the long run. But, no matter how 
large the group that collects the information, "people-problems" enter in. 
Ohe public, who increasingly demand proof of the effectiveness of services 
may also be the recipients of the service. Their voluntary cooperation is 
needed in its evaluation. Yet, many citizens resists the intrusion of 
surveys and questionnaires upon their privacy. Librarians, in many cases, 
are even less willing to ask the necessary questions. It may be assumed 
that librarians, who work with data every day would be ccmfbrtable collecting 
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it. But himan factors provide inconsistency to this assinTption. "Because 
I resist being questioned, I assume everyone else does." "It forces me to 
assume a raore personal role with the patron which is unconfortable for me." 
Or the opposite, "Keeping records on the services gets in the way of 
providing the service on a personal level." 

It can be said that understandir^ the reasons and techniques of data 
collection and 'involvement at ground level of all staff will prevent resistance 
to it. It is imperative that planners constantly strive for this understanding. 
It is also necessary,* however, that planners be cognizant of different value 
sy stents held, or time factors, or chaix^.^s in personnel v^iich any research 
in an action setting en\mls. Additionally, planners must realize the 
politics at play between different levels of data collection. Jbr exairple, 
is the person responsible . for overseeing measurement of a service also in 
chcurge of the budget for the departments in which the original data is 
collected? 

Ihe data obtained for the purpose of measuring performance must be used. 
Data collection and continuous analysis of it to sixM viiat is actually 
happening provide the information with management, or other decision 

makers, can evaluate the service. In addition, if the data has been dealt 
- with honestly throughout the collection points, one may find that v^t one 
thought was happening isn't at all. Thus, information needed to iirprove 
the service frcm an organizational viewpoint is available. The data may alert 
roanagannent to differences in staff philosophies ar^ value systems which effect 
the kind of data collected or the way in v^ch it is collected, and by 
extension, the kind of service beii^ provided. 
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In essence, the people, politics, and value systems all contribute 
to any honest appraisal of measures of perfonnance. Setting a target, 
however arbitrary, and assessing the degree to vMch it has been achieved 
demands an unavoidable "discussion of possible changes to correct deficiencies 
in the program. " Evaluation is the logical extension of performance measure- 
ment and can donand continuous changes, small or large. Joseph VJholey states 
that the "primary evaluation pay-off (in terms of decisions actually influenced) 
may be in evaluation that is done in enough detail to get at the effects of 
operational changes within operating programs."^ This assumes that manage- 
ment decision makers really want to know vAiat works best to provide the 
best service for the patron. Within this need to know, performance measures 
provide useful criteria for the evaluation of real outcaies~is the service 
meeting a need? Mditionally, they provide guidelines for a re-examination 
of the needs of patrons. This cxa-rination leads to new urderstanding of the 
desirability of the goals and objectives and makes one accutely aware of 
the chances of meeting them. i 



ERIC 



9, 



POOINOTES 

1. DeProspo, Ernest R. , A Prpgram Plaiininq and Evaluation Training 
Manual , College Entrance Examination Board, New York, 1973, p. 12. 

2. Ibid, p. 15. ■ 

3. Ibid, p. 2. 

4. Wildavsky, Aaron, "If Planning is Everything, Maybe It's Nothing", 
Policy Sciences , Vol. 4, 1973, p. 129. 

5. Mascarenhas,.R. C, "A Systans View of the Measurement of the 
Performance of Public Enterprises in Iivaia", Policy Sciences, Vol. 5, 1974. 
p. 29. ■ ^ 

6. DeProspo, E. , A Preliminary Draft of Evaluation Design and 
Data Systan Manual , July, 1974, p. 15. 

7. Suchman, Edvard A., Evaluative Research , Russell Sage FdOndation, 
New York, 1967, p. 162. 

8. VJholey, Joseph S., "What Can We Actually Get Fran Proaram 
Evaluation", Policy Sciences , Vol. 3, 1972, p. 365. 



ERIC 



10 



